Content-Based Retrieval from Medical Image Databases: A Synergy of Human Interaction, Machine Learning and Computer Vision

Content-based image retrieval (CBIR) refers the ability to retrieve images on the basis of image content. Given a query image, the goal of a CBIR system is to search the database and return the n most visually similar images to the query image. In this paper, we describe an approach to CBIR for medical databases that relies on human input, machine learning and computer vision. Specifically, we apply expert-level human interaction for solving that aspect of the problem which cannot yet be automated, we use computer vision for only those aspects of the problem to which it lends itself best image characterization and we employ machine learning algorithms to allow the system to be adapted to new clinical domains. We present empirical results for the domain of high resolution computed tomography (HRCT) of the lung. Our results illustrate the efficacy of a human-in-theloop approach to image characterization and the ability of our approach to adapt the retrieval process to a particular clinical domain through the application of machine learning algorithms.